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ABSTRACT 

Fourier analysis is used to remove periodic errors from a series of NIMBUS-5 
ESMR microwave brightness temperatures. The observations were all taken from 
the midnight orbits over fixed sties in the Australian grasslands during 1 974. The 
angular dependence of the data indicates calibration errors consisting of broad 
sidelobes and some miscalibration as a function of beam position. Even though 
an angular recalibration curve cannot be derived from the available data, the sys- 
tematic errors can be removed with a spectral filter. The 7-day cycle in the 
drift of the orbit of NIMBUS-5, coupled to the look -angle biases, produces an 
error pattern with peaks in its power spectrum at the weekly harmonics. About 
±4“K of error is removed by simply blocking the variations near two- and three- 
cycles-per-week. 
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Chapter I 
INTRODUCTION 


A series of ESMR (Electronically Scanned Microwave Radiometer) brightness temperatures 
were gathered by Allison et al.,^ at several sites in the desert and grasslands of eastern Australia, 
observed from NIMBUS-S during the first half of 1974. They were studying severe flooding 
there, using the steady summer data from the grass-covered areas as a baseline. Data was taken 
from each midnight pass to avoid the diurnal heating cycle. However, the ESMR data were far 
more variable than the corresponding surface reports. This is the opposite to what would be ex- 
pected of mss-covered ground, where the vegetation canopy should mask and buffer any changes 
in the ground’s microwave appearance. This study shows how to analyse and correct a sporadi- 
cally observed series of artificially flickering data without using any other independent observations. 

Sporadic problems in the ESMR data-processing chain disqualified one-third of the 120 
possible observations at each site. Data from three of the grassland sites is shown in Figure 1, 
where the few available sequences of consecutive midnight observations are connected by straight 
lines. The variations shown in Figure 1 are rapid and large, even during the quiet summer months 
of early 1974. The ESMR data show alternating warm and cold days, correlated among the sites, 
but not with the local meteorology. 

Table 1 lists the cross-correlations between variations in the ESMR temperatures at Cobar 
and several local meteorological parameters. The modest association of ESMR bnghtness with air 
temperature and soil moisture can be accounted for by the general seasonal changes and the au- 
tumnal flooding episode. No significant correlation was found between the flicker in the ESMR 
brightness and local meteorology, even on a delayed basis. Table 1 also lists the mean and standard 
deviation of each parameter, including the ESMR brightness temperature itself, which has a nom- 
inal instrumental noise of ±2°K. 
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Table 1 

ESMR Brightness at Cobar Correlated with Meteorology 



Parameter 

Mean ± Standard Deviation 

Correlation/ESMR 

Air Temperature -v®C) 

18.4 ± 4.4 

+0.58 

Dew Point (°C) 

13.4 ± 3.6 

+0.12 

Cloud Cover (Eighths) 

3.1 ± 3.1 

-0.29 

Daily Rainfall (mm) 

2.5 ± 8.2 

-0.16 

Soil Moisture (%)* 

44.0 ± 27.0 

-0.41 

ESMR Brightness Temp. (°K) 

258.0 ± 8.0 

- 


^interpolated from weekly reports 


Power spectra were taken of the sporadic series of ESMR temperatures shown in Figure 1. 

Sharp peaks at 7-day harmonics appeared, which seem related to the weekly drift of NIMBUS-5’s 
» rbit in association with regular angle-dependent sidelobc errors. 

Rather than doing a complete error analysis of the ESMR data processing chain, we filtered 
out the obviously artificial periodic variance in the available data, using the data itself as a guide. 

The purpose of this note is to describe the key ESMR data characteristics and to outline the 

method for correcting the periodic errors in an irregularly sampled series of observations. ' 







Chapter 2 

THE ESMR DATA REDUCTION 


Figure 1 shows the ESMR temperatures gathered at three sites: Condobolin, Cobar and 
Bourke, which are located a few hundred kilometers (a few ESMR beam positions) apart in the 
arid eastern Australian grasslands, as described by Allison, et al. ^ Because of some unreliability in 
the map coordinates assigned to the NIMBUS-S ESMR data, they renavigated each swath by eye 
to tit the islands and coastlines. When sites fell between nominal ESMR beam-positions, average 
values of the surrounding p:\els were used. 

Data was taken within a 120-day period from Australian mid-summer (20 January) to late 
autumn (19 May) 1974. All three sites experienced severe flooding in autumn (April) 1974, while 
Bourke also was flooded during summer (February) 1974. This appears as lower ESMR brightness 
temperatures in Figure 1, due to the low microwave emissivity of water. About 69 of the possible 
1 20 nights have usable ESMR data. 

Calibration of the ESMR brightneos temperatures was done by the GSFC ERTS/Nimbus 
Project^ using their linear recalibration algorithm for each beam position.^ Their data processing 
software itself undergoes occasional unpublished revisions. 

Figure 1 shows strong correlations in the short period fluctuations of the observed ESMR 
temperatures among the grassland sites. Table 2 lists the correlations between pairs of sites, be- 
fore and after filtering of the site observations. Cobar and Condobolin are probably more correlated 
with each other than with Bourke because they lie close together on similar high, dry grassland, 
while Bourke is on lightly flooded, stream-fed ground. Indeed, the ESMR observations at Bourke 
run systematically colder by about 5°K throughout the summer. 

Tlie nadir-angle of each observation was estimated by overlaying a Mercator projection of the 
ESMR beam ground tracks, drawn for Australian latitudes from the nominal orbit description for 
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Table 2 

ESMR Correlations Between Sites 
Before/ After Filtering 


Site/Site 

Before 

After 

Cobar/Bourke 

0.58 

0.17 

Cobar/Condobolin 

0.71 


Bourke/Condobolin 

0.73 



NlMBUS-5. The angles were estimated by aligning the projection map with the actual swath, to 
an accuracy of a few degrees of longitude (a few beam positions). Figure 2 shows the ESMR tem- 
peratures and corresponding nadir-angies for midnight observations at one site for two months of 
steady dry summer weather. Note that the orbit is arranged so that if one night’s pass looks to 
the east of nadir, the next night’s usually looks to the west. To avoid errors at the edge of the 
ESMR swaths, observations at nadir angles beyond 40° (out of a possible 50°) were not used. 

Figure 3 is a regression plot between these ESMR temperatures and the corresponding nadir 
angles at all three sites for the quiet summer data. The linear regression lines in Figure 3 indicate 
a 10°K difference in viewing the same site from the east versus the west. The warmer tempera- 
tures recorded when ESMR looked to the east of nadir are interpreted as due to broad sidelobes 
being filled by the bright Australian continent directly below, as opposed to the cooler tempera- 
tures reported when ESMR looked west from over the dark ocean. There are also distinct angle- 
dependent miscalibrations indicated, with roughly a '‘skewed W” pattern of being high at the 
center and edges, and low at 20° from nadir. Figure 3 can show the main features of the infer- 
red nadir-angle dependence of the Australian ESMR temperatures, but it cannot serve as an angu- 
lar recalibration curve. Fortunately, die data in Figure 1 arc good enough to Fourier analyse and 
correct for nearly all of the per odic errors without additional information. 
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Chapter 3 

FOURIER ANALYSIS OF SPORADIC DATA 

Although computer oriented formulations of Fourier fllteriug techniques are now commonly 
used, the conventions are not yet standardized. Loosely following Brigham,^ this section casts 
the basic equations in practical digital form. 

The familiar abstract analog form of the Fourier transform pair g(t) and G(f) for time (t) and 
frequency (0 is shown in Equation (3.1). 

fOO fOO 

G(0 = / dtg(t)e-2ffift, g(t) = / df(K0e-^2jrift (3.1) 

•'-00 •'-00 

The corresponding indexed series g<k) = g(kT) of N discrete measurements, taken at regular time 
intervals T starting at zero, has a Fourier transform series G<n) = G(n/NT), shown in Equation 
(3.2). 

N-l N-1 

G(n> = g<k>e-2nikn/N, 8<k> = TT G(n> e*2nikn/N (3.2) 

k»o n«0 

where |k( = |n| = |o,l,2, . . . , N - l|. 

As usual, the high frequency cutoff was set at the sampling rate 1/T, and the steps in the 
frequency scale were set equal to the bandwidth resolution limit allowed by the experiment dur- 
ation rate 1/NT. Equation (3.2) implicitly uses the sampling rate 1/T as the natural rate units. 

For instance, sampling once per day produces frequency units of cycles per day, so that n/N - 3/7 
means “three cycles per week.” Alternative formulations for Fourier series must be made care- 
fully in order to avoid errors in the normalization and accounting conventions. In particular, the 
transformation basis exp (±2nikn/N) is the only complete orthogonal pair (with a normalization 
constant equal to N) for the positive indices |k| and |n|. The complex notation for G is actually 
computed as two parallel series of numbe»s, the cosine (real) and sine (imaginary) transformation 
coefficients. Theoretically, N independent complex numbers determine N pairs of transform 
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coefficients; actually, N raid observations reduce the number of independent values in the complex 
coefficients imposing the constraint G<n>**G<N • n). 


Equatimi (3.3) defines the low-frequency integrated power P[G) in terms of the power 
den^ty spectrum iGl^. 


j n<N 

P<n)(Gl y\ lG<n'>|2 (3.3) 

N ^ 
n«0 

Note that lG<n>l*» IG<N - .i>|2. 

Because the norm IGI^ destroys phase information in the power spectrum, it is uniquely de> 
termined only for positive frequencies up to one-half of the sampling rate (n/N< 1/2). The steps 
in a plot of the integrated power show the fraction of power contained in each resonance peak of 
the spectrum; the total power is P<N)s^ lg<k> 1^. In graphical analysis, power is usually shovm 
normalized to the hipest peak for spectra, and normalized to the total for accumulated power. 


The missing values in a series of measurements are treated as observations of zero weight. 
Mathematically, a template function g' is created, with value 1 at the time of a good observation, 
and value 0 at a missing observation. Then, the series of sporadic observations is filled in by using 
the product g'g** of the template g' and an ideal series of data g^. Equation (3.4) shows this 
construction and the product/convolution theorem for the Fourier transform of such a product 
of functions. 


g<k> = g’<k>gO<k>, 



(3.4) 


using G’<-m>* G'<m> for jm{ ■ jo,l,2, . . . ,N - l( . 


Ginvolution blur-i the ideal transform with the bandpass function G'. Completely regular 
observations have a strong, sharp bandp.ass set by the experiment duration rate. While the blur 
cannot be removed without more information, its power spectrum iG'i^ shows how the pattern 
of missing observations v/ill broaden or echo the peaks in an idea! spectrum IG*’!*. For instance, 
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a pattern of regularly musing observations will gene ;te sideiobes from a real peak, due to the 
frequency ambiguity at the absent cycle rate. A pattern of randomly missing observations merely 
broadens all features in the power spectrum, due to the absence of some information at all regular 
sampling rates. The power spectrum of the observation template IG* P can be termed the “leakage* 
in the information spectrum. Basically, some information averaging of less than N independent 
observations must be done to create N Fourier coefficients. 

Sophisticated filters are used when the quality of the data, the nature of the errors, and the 
data application requirements are all well known. However, with only crude artificial peaks as a 
guide, the appropriate filter is one that simply blocks all power at the suspected frequency. Be- 
cause the power spectrum is phase-ambiguous and the filtered result should remain a series of real 
numbers, the Fourier coefficients must be removed at the conjugate frequency as well (using 
G(N-m)-G(m>* for real g<k». Equation (3.5) shows this worked out to produce a new time 
series g<k;m) with little power at frequency m/N cycles per sample-time, and no variance at the 
missing observations. 

g<k;m> = g'<k>|g(k> - (2/N)(Re(G<m>) cos(2jrkm/N) - Im(G<m)) sin(2irkm/N)l| (3.5) 
for frequency 0 < m/N < 1/2. 

The filter imposed by Euqation (3.5) first removes all variations at the suspected frequency. 
This creates lluctuations in the regularized series of filtered observations at the times when no 
observation was taken. When the observation template g' is used to suppress the induced fluctua- 
tions, the cleanly blocked power spectrum is again blurred according to the product/convolution 
theorem of Equation (3.4). Tlic effect is to level the final filtered spectrum a little, by spreading 
the available information over all N spectral points. If more than one frequency is to be blocked, 
all of the corresponding Fourier coefficients should be subtracted in Equation (3.5) before being 
multiplied by the template g'. Tlie blocking filter of Equation (3.5) removes all variations at the 
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suspected t'lcquency, including the natural ones. This loss amounts to only 2/N of the total 
natural variance, for a well-sampled series with a naturally flat power spectrum. 

The following steps outline the algorithm for removing most of the simple periodic errors in 
a series of data with missing observations; 

1. Select a sc()uence of observations that should he internally consistent. 

2. Subtract the average value to make a series of spi>radic variations g*', and create an ob* 
servation template g'. 

3. Create the regularly sampled series g = g'g*' by setline, the unooserved variations to /.cro 
value. 

4. Calculate and plot the normalized power spectra Ki'P and iCil^ to see how the mis.sing 
observations might distort the resonant peaks. 

5. Classify peaks as “artificial” if they are strong, are unrelated to natural physical causes, 
are unrelated to bandpass function distoritons of other peaks, and/or are near suspicious 
harmonics. 

6. Remove the “artificial” periodicities with Ixjuation (3.5) to form a filtered sporadic series 
g<k;m.in',. . .). 

7. Compare the integrated power l’K*l befiue and after filtering, and estimate the amount 
ci erroneous variutue removed. 

H. Add the original mean value back to tl.e tillered scries for subsequent scientific analysis. 




Chapter 4 

FILTERING THE ESMR DATA 

We will step througli the filtering algorithm for the 1974 summer Cobar scries of ESMR 
brightness temperatures shown in Figure 2. and then tabulate the effect of similar filters for the 
other cases shown in Figure I . 

Power spectra IG and the corresponding bandpass function IG' 1^ of the uniformly re* 
sampled FSMR variations from Figure 2 are shown in Figure 4. Even thougli ther- arc only 41 
good observations in 62 days, the bandpass IG'I^ is well behaved, with only minor sidebands at 
1/7 and 3/7 (once and thrice per week) due to some regularity in the missing observations tno 
S;.turdays). However, the *:SMR variations themselves show strong artificial peaks near frequencies 
of 2/7 and 3/7, and a natural low-frequency drift. The association of the artificial ESMR tem- 
perature periodicities with angular biases is strengthened by the similarity between their power 
spectra (compare Figures 4 and 5). 

Figure 6 shows that one-third of the total variance in the FSMR temperatures is removed by 
applying blocking filters near the artificial frequencies 2/7 and 3/7 (actuallv at m/N« 17/62 and 
26/62). Specifically, the standard deviation of the sporadic series decreased from ±5.5*K to 
t4.3"K, removing ±3.4®K. Comparison of Figures 3 and 7 shows that most of the inferred cor- 
relation between I.SMK temperature and scan angle is removed by the blockage of the weekly 
periodicities. 

Table 3 lists the improvements in the standard deviations of the FSMR temperatures at all 
three grassland sites in summer and autumn. The residuals after filtering are white noise and long 
period drills, in about equal amounts. Ihe seasons are handled separately not only becuase of 
the climatic differences, hut also because of the large gap in the FSMR observations at the change 
of seasons. Note that an orbital change would require the start of a new filter, because a phase- 
shift induced by the change would violate the simple periodicity assumed in the blocking filter. 
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Table 3 

ESMR Standard Deviations Before/ After Filtering 


Season 

Site 

Before 

After 

Removed 

summer 

Cobar 

±5.5°K 

±4.3®K 

±3.4“K 

summer 

Condobolin 

±6.0“K 

±3.6“K 

±4.8“K 

su»"mer 

Bourke 

±6.7"K 

±5.5“K 

±3.8®K 

autumn 

Cobar 

±6.7®K 

±5.TK 

±4.4“K 

autumn 

Condobolin 

±7.1°K 

±5.6“K 

±4.4-’K 

autumn 

Bourke 

±6.6°K 

±5.3°K 

±3.9“K 


Finally, comparison of Figures 8 and 1 shows the improvement in all series of ESMR temperatures 
when the weekly harmonics are blocked. For each of these cases, the periodic errors added 
about ±4“K, due mainly to the large east-west bias from the bright Australian continent filling 
broad sidelobes in the ESMR antenna pattern. Note that the flooding episode in the autumnal 
observations is actually clarified by the filtering, which was, after all, the original purpose in de- 
vising a correction scheme for Allison’s* 1974 Australian ESMR data. 
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Qiapter 5 

COMMENTS ON ESMR DATA 

Many of the missing ESMR observations were rejected because the data swaths showed “glitch 
mode” pattems-streaks of artificially high or low brightness running parallel to the satellite ground 
track. In fact, the gap in the Australian ESMR data in April 1 974 was due to a continuous “glitch 
mode” episode. It seems likely that the low temperatures frequently recorded at 20® to cither 
side of nadir are a subtler form of this. 

The observations which were taken at the Australian desert sites from the same ESMR swaths 
were too few and too biased toward the swath’s edge for good filter analysis. They did show con- 
siderably more variance, with crude periodic errors having weekly harmonics and angle-dependence. 

The Australian ESMR data were reduced in ways aimed to avoid the foreseeable errors. The 
normal diurnal variations in ground temperature were minimized in the ESMR temperatures by 
the use of only midnight observations. Faulty geographic positioning of the ESMR data was cor- 
rected by the renavigation of each image with the coastline of eastern Australia. Variations due 
to the details of the local geography should be minimized by the fact that the sites are located in 
a homogeneous area. The use of grassland sites, whose microwave appearance is buffered by the 
vegetation, and the lack of correlation with local meteorology indicates that the periodic errors 
are essentially instrumental. 

Current users of NlMBUS-5 ESMR data do not find problems of exactly our type, but they 
do not work with months of ESMR data on a bright continent that fills much of the potential 
sidelobes. For instance, Kidder^ corrected calibration insensitivity at beam positions near small 
and large nadir angles by requiring steady averages from the quiet open tropical ocean. Likewise, 
Carsey^ rejects data containing significant dips in running-mean values of the ESMR brightness 
of the open Antarctic Ocean. Without undertaking a major project to analyse ESMR errors, we 
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are unable to prove whether or not the periodic (nadir-angle) errors are really unique to the 1974 
Australian data, or are merely enhanced by the unusual use of it. 

Future users of ESMR data should be supplied with a technical summary of the antenna 
pattern, of known problems, and of the results of verification experiments. If another ESMR is 
ever launched, a thorough pre-launch calibration of the antenna pattern for each beam position 
is needed to understand for the sidelobes. Otherwise, good quantitative work near the bright/ 
dark land/sea boundaries is impossible. 
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FIGURES USED 
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Figure 2. Summer ESMR Temperature and Nadir-Angle at Cobar 
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Figure 3. Summer ESMR Temperature vs. Nadir-Angle at Three Sites 


15 





Figure 4. Power Speetrum of S 
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Figure 6. Integrated Power for Summer Cobar Before/After Filtering 
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Figure 8. Filtered Series of ESMR Temperatures at Three Sites 
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